Preheat oven to 350. Butter 8x8x2 inch baking dish. Beat together butter and sugar until well blended. Stir in sweet potatoes,milk,brandy,orange rind,ginger.cinnamon, and nutmeg. Spoon into prepared pan. Bake at 350 for 1 hour or until set and lightly crisp on top. Serve warm.